简介

Elastalert是用python2写的一个报警框架(目前支持python2.6和2.7，不支持3.x)，github地址为。他提供不同场景的规则配置，若觉得规则、告警不满足需求时，可以用python编写插件、。

Elastalert内置的告警方式：

JIRA

OpsGenie

Commands

HipChat

MS Teams

Slack

AWS SNS

VictorOps

PagerDuty

Exotel

Twilio

Gitter

前置条件

Elasticsearch

Python 2.7

pip 是 Python 包管理工具，该工具提供了对Python 包的查找、下载、安装、卸载的功能。目前如果你在 python.org 下载最新版本的安装包，则是已经自带了该工具。Python 2.7.9 + 或 Python 3.4+ 以上版本都自带 pip 工具。pip 官网：https://pypi.org/project/pip/你可以通过以下命令来判断是否已安装：pip --version如果你还未安装，则可以使用以下方法来安装：$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py   # 下载安装脚本$ sudo python get-pip.py    # 运行安装脚本注意：用哪个版本的 Python 运行安装脚本，pip 就被关联到哪个版本，如果是 Python3 则执行以下命令：$ sudo python3 get-pip.py    # 运行安装脚本。一般情况 pip 对应的是 Python 2.7，pip3 对应的是 Python 3.x。部分 Linux 发行版可直接用包管理器安装 pip，如 Debian 和 Ubuntu：sudo apt-get install python-pip

sudo easy_install -U setuptools

sudo yum install python-devel

python-pip python-dev libffi-dev libssl-dev

安装

直接使用pip安装：

$ pip install elastalert  直接装的官网发布的，不好用，还是下载源码安装

或者克隆源码安装：

$ git clone https://github.com/Yelp/elastalert.git

安装模块：

$ pip install "setuptools>=11.3" or sudo ....$ python setup.py install

根据不同的Elasticsearch版本安装elasticsearch-py。

Elasticsearch 5.0+:

$ pip install "elasticsearch>=5.0.0"

Elasticsearch 2.X:

$ pip install "elasticsearch<3.0.0"

补充：

若出现如下问题：

error: six 1.9.0 is installed but six>=1.10.0 is required by set(['jira'])

解决办法：

$ pip install --upgrade six

更新的位置不是 python lib找的位置：/usr/lib/python2.7/site-packages/pkg_resources            /root/.local/lib/python2.7/site-packages1，sudo pip install --upgrade sixDEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.Requirement already up-to-date: six in /root/.local/lib/python2.7/site-packages (1.12.0)2，elastalert-create-indexTraceback (most recent call last):  File "/usr/bin/elastalert-create-index", line 6, in 
       
            from pkg_resources import load_entry_point  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3241, in 
        
             @_call_aside  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3225, in _call_aside    f(*args, **kwargs)  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3254, in _initialize_master_working_set    working_set = WorkingSet._build_master()  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 585, in _build_master    return cls._build_from_requirements(__requires__)  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 598, in _build_from_requirements    dists = ws.resolve(reqs, Environment())  File "/usr/lib/python2.7/site-packages/pkg_resources/__init__.py", line 791, in resolve    raise VersionConflict(dist, req).with_context(dependent_req) 解决方法：

pip uninstall  six pip install --upgrade six

卸载后重新安装就行。

  然后继续安装即可。
 如果在windows 64平台报错：执行

pip install python-magic-bin==0.4.14 修复https://stackoverflow.com/questions/18374103/exception-valuefailed-to-find-libmagic-check-your-installation-in-windows-7

E:\elastalert\example_rules> python -m elastalert.elastalert --verbose --rule config.yamlTraceback (most recent call last):  File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main    "__main__", fname, loader, pkg_name)  File "C:\Python27\lib\runpy.py", line 72, in _run_code    exec code in run_globals  File "C:\Python27\lib\site-packages\elastalert\elastalert.py", line 21, in 
       
            from alerts import DebugAlerter File "C:\Python27\lib\site-packages\elastalert\alerts.py", line 31, in 
        
          from thehive4py.api import TheHiveApi File "C:\Python27\lib\site-packages\thehive4py\api.py", line 7, in 
         
           import magic File "C:\Python27\lib\site-packages\magic.py", line 181, in 
          
            raise ImportError('failed to find libmagic. Check your installation') ImportError: failed to find libmagic. Check your installation

重命名配置文件：

$ cp config.yaml.example config.yaml

打开配置文件config.yaml：

$ vim config.yaml

设置es_host、es_host指向你的Elasticsearch，然后保存退出。

配置文件解析

此处的配置文件是指config.yaml文件。

rules_folder：ElastAlert将加载规则配置文件的地方，它将尝试加载文件夹中的每个.yaml文件。

run_every：ElastAlert查询Elasticsearch的频率。

buffer_time：是查询窗口的大小，从每个查询运行的时间向后延伸。对于其中use_count_query或use_terms_query设置为true的规则，此值将被忽略。

es_host：是Elasticsearch集群的地址，ElastAlert将存储有关其状态、查询运行、警报和错误的数据。每个规则也可以设置不同的elasticsearch主机进行查询。

es_port：Elasticsearch对应的端口。

use_ssl：（可选的）是否使用TLS;连接到es_host；设置为True或False。

verify_certs：（可选的）是否验证TLS证书; 设置为True或False，默认是True。

client_cert：（可选的）PEM证书的路径。

client_key：（可选的）作为客户端密钥使用的私钥文件的路径。

ca_certs：（可选的）用于验证SSL连接的CA证书的路径。

es_username：（可选的）用于连接Elasticsearch的basic-auth用户名。

es_password：（可选的）用于连接Elasticsearch的密码。

es_url_prefix：（可选的） Elasticsearch端点的URL前缀。

es_send_get_body_as：（可选的）查询Elasticsearch方法- GET，POST或source，默认是GET。

writeback_index：是ElastAlert将存储数据的索引名称。

alert_time_limit：是失败警报的重试窗口。

创建Elasticsearch索引

elastalert-create-index这个命令会在elasticsearch创建索引，便于ElastAlert将有关其查询及其警报的信息和元数据保存回Elasticsearch。这不是必须的步骤，但是强烈建议创建。因为对于审计，测试很有用，并且重启elastalert不影响计数和发送alert。默认情况下，创建的索引叫 elastalert_status。

$ elastalert-create-indexNew index name (Default elastalert_status)Name of existing index to copy (Default None)New index elastalert_status created Done!

创建成功之后，我们可以在Elasticsearch中看到所创建的索引elastalert_status：

$ curl 'localhost:9200/_cat/indices?v'health status index             uuid                   pri rep docs.count docs.deleted store.size pri.store.sizeyellow open   .kibana           _4Op-xRCSzyC5nI2wZSPvw   1   1          2 0 8.4kb 8.4kb yellow open elastalert_status ZTetmp8mTS6F28aIBJmJOg 5 1 0 0 810b 810b

elastalert_status

ElastAlert 根据elastalert_status去确定首次启动的时候在什么时间范围内去查询，以避免重复查询。对于每个规则，它将从最近的结束时间开始查询。包括：

@timestamp：文件上传到Elasticsearch的时间。这是在运行查询并且已经处理结果之后。

rule_name：相应规则的名称。

starttime：查询的开始时间戳。

endtime：查询结束时间戳。

hits：查询结果的数量。

matches：处理命中后规则返回的匹配数。请注意，这并不一定意味着警报被触发。

time_taken：此查询运行所需的秒数。

测试Demo

本次elastalert安装目录为：/opt/elastalert。

配置SMTP

在/opt/elastalert目录下创建文件smtp_auth_file.yaml，内容如下：

#邮箱用户名user: example@email.com#不是邮箱密码，是设置的SMTP密码password: XXXXX

配置规则

在example_rules目录下修改配置文件，如下：

$ cd /opt/elastalert/example_rules$ vim example_frequency.yaml

修改后的配置文件：

# Alert when the rate of events exceeds a threshold# (Optional)# Elasticsearch host# es_host: elasticsearch.example.com # (Optional) # Elasticsearch port # es_port: 9200 # (OptionaL) Connect with SSL to Elasticsearch #use_ssl: True # (Optional) basic-auth username and password for Elasticsearch #es_username: someusername #es_password: somepassword # (Required) # Rule name, must be unique name: Example rule # (Required) # Type of alert. # the frequency rule type alerts when num_events events occur with timeframe time type: frequency # (Required) # Index to search, wildcard supported index: logstash-2017.08.28 # (Required, frequency specific) # Alert when this many documents matching the query occur within a timeframe num_events: 1 # (Required, frequency specific) # num_events must occur within this amount of time to trigger an alert timeframe: minutes: 1 # (Required) # A list of Elasticsearch filters used for find events # These filters are joined with AND and nested in a filtered query # For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html #匹配日志中field字段值为value的日志数据，若不想进行过滤可以写成“filter: []”（这种方式可以用在日志数量的告警中） filter: - query: query_string: query: "field: value" #SMTP configration smtp_host: smtp.163.com smtp_port: 25 #SMTP auth smtp_auth_file: /opt/elastalert/smtp_auth_file.yaml email_reply_to: example@email.com from_addr: example@email.com # (Required) # The alert is use when a match is found alert: - "email" # (required, email specific) # a list of email addresses to send alerts to email: - "test@email.com"

保存修改并退出。

补充：

上述规则表示：在elastalert执行的一分钟内，出现一条有field值为value日志，则触发告警，并且告警通知将以email的形式从example@email.com邮箱发送给test@email.com。

规则配置解析：

es_host、es_port：应该指向我们要查询的Elasticsearch集群。

name：是这个规则的唯一名称。如果两个规则共享相同的名称，ElastAlert将不会启动。

type：每个规则都有不同的类型，可能会采用不同的参数。该frequency类型表示“在timeframe时间内匹配成功次数超过num_events发出警报”。有关其他类型的信息，请参阅。

index：要查询的索引的名称。

num_events：此参数特定于frequency类型，是触发警报时的阈值。

timeframe：timeframe是num_events必须发生的时间段。

filter：是用于过滤结果的Elasticsearch过滤器列表。有关
详细信息，请参阅。

alert：警报。有关警报类型的更多信息，请参阅。电子邮件警报需要SMTP服务器才能发送邮件。默认情况下，它将尝试使用localhost。这可以通过smtp_host选项更改。

email：是要发送警报的地址列表。

还有许多其他可选配置选项，请参阅。

myrule.yaml和config.yaml

# From example_rules/example_frequency.yamles_host: xxx.xxxes_port: 9200name: Example ruletype: frequencyindex: myindextimestamp_field: mytimenum_events: 1timeframe:  minutes: 50filter:- term:    "carColor" : "blue"alert:- "email"email:- "elastalert@example.com"

# This is the folder that contains the rule yaml files# Any .yaml file will be loaded as a rulerules_folder: example_rules# How often ElastAlert will query Elasticsearch# The unit can be anything from weeks to secondsrun_every:  minutes: 1# ElastAlert will buffer results from the most recent# period of time, in case some log sources are not in real timebuffer_time:  minutes: 15# The Elasticsearch hostname for metadata writeback# Note that every rule can have its own Elasticsearch hostes_host: 211.100.75.204# The Elasticsearch portes_port: 9200# The AWS region to use. Set this when using AWS-managed elasticsearch#aws_region: us-east-1# The AWS profile to use. Use this if you are using an aws-cli profile.# See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html# for details#profile: test# Optional URL prefix for Elasticsearch#es_url_prefix: elasticsearch# Connect with TLS to Elasticsearch#use_ssl: True# Verify TLS certificates#verify_certs: True# GET request with body is the default option for Elasticsearch.# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport# for details#es_send_get_body_as: GET# Option basic-auth username and password for Elasticsearch#es_username: someusername#es_password: somepassword# Use SSL authentication with client certificates client_cert must be# a pem file containing both cert and key for client#verify_certs: True#ca_certs: /path/to/cacert.pem#client_cert: /path/to/client_cert.pem#client_key: /path/to/client_key.key# The index on es_host which is used for metadata storage# This can be a unmapped index, but it is recommended that you run# elastalert-create-index to set a mappingwriteback_index: elastalert_status# If an alert fails for some reason, ElastAlert will retry# sending the alert until this time period has elapsedalert_time_limit:  days: 2

更全的示例：

# Alert when the rate of events exceeds a threshold# (Optional)# Elasticsearch hostes_host: 110.10.21.77# (Optional)# Elasticsearch portes_port: 9200# (OptionaL) Connect with SSL to Elasticsearch#use_ssl: True# (Optional) basic-auth username and password for Elasticsearch#es_username: someusername#es_password: somepassword# (Required)# Rule name, must be uniquename:  system error # (Required)# Type of alert.# the frequency rule type alerts when num_events events occur with timeframe timetype: frequency# (Required)# Index to search, wildcard supportedindex:  task*attach_related: true# (Required, frequency specific)# Alert when this many documents matching the query occur within a timeframenum_events: 1# (Required, frequency specific)# num_events must occur within this amount of time to trigger an alerttimeframe:  hours: 1timestamp_field: "@timestamp"# (Required)# A list of Elasticsearch filters used for find events# These filters are joined with AND and nested in a filtered query# For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.htmlfilter:- terms:    "Level": ["fatal", "error"]#use_kibana_dashboard: afAlertDashboarduse_kibana4_dashboard: https://nodejsgbl.italkbb.com/kibana/app/kibana#/dashboard/3bde48d0-9880-11e9-b5d5-2df46b09dea6# (Required)# The alert is use when a match is foundalert:- "email"# (required, email specific)# a list of email addresses to send alerts toemail:- "xxxx@net263.com" - "xxxx@163.com"smtp_host: smtp.263.netsmtp_port: 25smtp_auth_file: ../smtp_auth_file.yamlemail_reply_to: No.reply@net263.comfrom_addr: ElastAlert@net263.com#cc: aaa@net263.comemail_format: html#alert_text_type: exclude_fields #alert_text_onlyalert_subject: "Alert: System {0} occurred {1} times."alert_subject_args:- Level #来自index里面json内容- "num_hits"alert_text: " You can access all of error log on Kibana:  Aijia server alert dashboard
"alert_text_args:- kibana_link

{"indexId":"1000001","mytime": "2019-01-14T17:40:17+08:00","carColor":"blue","@timestamp":"2018-12-18T07:48:17","cost":1001,"indexName":"default_log_index" }{ "indexName" : "af-test",     "code" : "1",          "msg" : "testMsg",          "position" : "AHC.User.ReadService",          "eventType" : "alarm","eventTimestamp" : "2019-01-14T17:40:17+08:00",          "@timestamp" : "2019-01-14T17:44:24.6172215+08:00"}key要区分大小写，value 写过滤条件时候，全部小写，否则查询不到。如 term: "msg":"testmsg" 才行

测试规则

命令如下：

$ elastalert-test-rule example_rules/example_frequency.yaml

能够检查规则是否正确，有关详细信息。

运行

运行elastalert：

$ cd /opt/elastalert$ python -m elastalert.elastalert --verbose --rule example_frequency.yaml 或者运行指定目录下所有yaml文件：  elastalert --verbose --config ./config.yaml  --start 2019-06-27T06:00:03 其中config.yaml里面存放着rules的目录。仔细看好目录名称不要写错。 调试elasticsearch 的 查询 语句是否有问题，可以加上参数 es_debug、es_debug_trace trace后面接文件名，里面会输出查询语句。

trace文件打印出查询语句： curl -XGET 'http://localhost:9200/?pretty' -d ''  "query": {    "bool": {      "filter": {        "bool": {          "must": [                  "gt": "2019-07-03T09:30:38.664486Z",                  "lte": "2019-07-03T09:32:03.508890Z"                }              }            },            {              "query_string": {                "query": "NOT AggregatedHealthState.keyword: Ok"              }            }          ]        }      }    }  },  "sort": [    {      "@timestamp": {        "order": "asc"      }    }  ]}

给Elasticsearch添加测试数据：

$ curl -X POST "http://127.0.0.1:9200/logstash-2017.08.28/test"  -d '{"@timestamp": "2017-08-28T10:54:41.000Z","field": "value"}'

注意：上述的@timestamp是UTC时间，也就是说这条数据是在2017-08-28 18:54:41产生的。

Alert

可在邮箱中看到如下图所示的邮件：

收到邮件中会包含当前符合告警规则的日志数据。

参考资料

https://www.cnblogs.com/zhaishaomin/p/7417306.html

https://blog.csdn.net/pujiaolin/article/details/52252950?locationNum=3

https://blog.csdn.net/gamer_gyt/article/details/52917116

https://blog.csdn.net/davinciyxw/article/details/79104498

https://blog.csdn.net/vbaspdelphi/article/details/54311764

http://www.freebuf.com/sectool/164591.html

alert还有kibana插件,以及elasticalert还可以容器方式启动

参考:

https://github.com/bitsensor/elastalert-kibana-plugin

插件下载慢:

https://git.bitsensor.io/front-end/elastalert-kibana-plugin/-/jobs/10874/artifacts/raw/artifact/elastalert-5.6.4-latest.zip 然后 ./bin/kibana-plugin install file:///usr/local/src/elastalert-5.6.4-latest.zip

安装后效果:

微信报警：https://github.com/anjia0532/elastalert-wechat-plugin

钉钉报警：https://github.com/xuyaoqiang/elastalert-dingtalk-plugin

告警规则

告警模板

告警时间

告警方式

-----

参考:https://xizhibei.github.io/2017/11/19/alerting-with-elastalert/

https://github.com/chenryn/ELKstack-guide-cn/blob/master/elasticsearch/other/elastalert.md

安装elastalert（按照官网的操作走）

先切换到自己的工作目录

我这里是 /usr/local/dev/

git 克隆源码

git clone https://github.com/Yelp/elastalert.git

cd elastalert

安装

sudo python setup.py install

出错了

Running blist-1.3.6/setup.py -q bdist_egg –dist-dir /tmp/easy_install-Gc6gbe/blist-1.3.6/egg-dist-tmp-Ik7LL2

The required version of setuptools (>=1.1.6) is not available,

and can’t be installed while this script is running. Please

install a more recent version first, using

‘easy_install -U setuptools’.

(Currently using setuptools 0.9.8 (/usr/lib/python2.7/site-packages))

ok按照它提示的，，，执行：

sudo easy_install -U setuptools

再执行安装（sudo python setup.py install），又出错了

warning: no files found matching 'blist.rst'

blist/_blist.c:38:20: 致命错误：Python.h：没有那个文件或目录

#include<Python.h>

这其实是缺少开发包，执行命令：

sudo yum install python-devel

再执行安装（sudo python setup.py install）

到这里基本能安装成功。

有时候会报一些包找不到的错误，可能是网络问题，重复执行安装命令即可（我在阿里云上遇到了）。

继续执行安装命令

pip install -r requirements.txt

找不到pip ，先安装pip并更新

sudo yum -y install python-pip

sudo pip install --upgrade pip

安装完继续

sudo pip install -r requirements.txt

在elasticsearch中创建elastalert的日志索引

sudo elastalert-create-index

根据自己的情况，填入elasticsearch的相关信息，关于

elastalert_status部分直接回车默认的即可。

如下所示：

Enter elasticsearch host: log.example.com

Enter elasticsearch port: 9200

Use SSL? t/f: t

Enter optional basic-auth username (or leave blank): es_admin

Enter optional basic-auth password (or leave blank):

Enter optional Elasticsearch URL prefix (prepends a string to the URL of every request):

New index name? (Default elastalert_status)

Name of existing index to copy? (Default None)

/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html

InsecureRequestWarning)

New index elastalert_status created

Done!

3.创建配置文件

sudo cp config.yaml.example config.yaml

sudo vi config.yaml

根据自己的具体情况进行修改

比如我这里的elasticsearch是ssl 并需要用户登陆的，配置大致如下：

run_every:

minutes: 1

buffer_time:

minutes: 15

es_host: log.example.com

es_port: 9200

use_ssl: True

es_send_get_body_as: GET

es_username: es_admin

es_password: es_password

writeback_index: elastalert_status

alert_time_limit:

days: 2

以同样的方式配置规则

cd example_rules/

sudo cp example_frequency.yaml my_rule.yaml

sudo vi my_rule.yaml

这里就是根据具体的elasticsearch的信息进行配置

es_host: log.example.com

es_port: 9200

use_ssl: True

es_username: es_admin

es_password: es_password

#name属性要求唯一，这里最好能标示自己的产品

name: My-Product Exception Alert

#类型，我选择任何匹配的条件都发送邮件警告

type: any

#需要监控的索引，支持通配

index: logstash-*

#下面两个随意配置

num_events: 50

timeframe:

hours: 4

#根据条件进行过滤查询（这里我只要出现异常的日志，并且排除业务异常（自定义异常））

filter:

- query:

query_string:

query: "message: *exception* AND message: (!*BusinessException*) AND message: (!*ServiceException*)"

#email的警告方式

alert:

- "email"

#增加邮件内容，这里我附加一个日志访问路径

alert_text: "Ref Log https://log.example.com:5601/app/kibana"

#SMTP协议的邮件服务器相关配置（我这里是腾讯企业邮箱）

smtp_host: smtp.exmail.qq.com

smtp_port: 25

#用户认证文件，需要user和password两个属性

smtp_auth_file: smtp_auth_file.yaml

email_reply_to: no-reply@example.com

from_addr: no-reply@example.com

#需要接受邮件的邮箱地址列表

email:

- "user1@example.com"

接下来创建smtp_auth_file.yaml

sudo touch smtp_auth_file.yaml

sudo vi smtp_auth_file.yaml

##配置文件内容

user: "no-reply@example.com"

password: "password"

现在可以简单测试一下配置文件是否正确

sudo elastalert-test-rule ./my_rule.yaml

如果有问题，就检查配置文件。

然后启动运行一下试试：

sudo python -m elastalert.elastalert --verbose --rule my_rule.yaml

然后故意产生一下异常试试，比如

if(true){

throw new NullPointerException("测试日志异常监控服务");

}

执行一下，一般没啥问题，邮件应该就能收到了，有问题就检查配置文件。

4.我是要监控服务，所以要让elastalert以服务的形式运行

首先创建elastalert的配置目录和相关文件

sudo mkdir /etc/elastalert

cd /etc/elastalert

-- 复制配置文件

sudo cp /usr/local/dev/elastalert/config.yaml config.yaml

sudo mkdir rules

cd rules

-- 复制规则文件

sudo cp /usr/local/dev/elastalert/example_rules/my_rule.yaml my_rule.yaml

-- 复制邮件用户认证文件

sudo cp /usr/local/dev/elastalert/example_rules/smtp_auth_file.yaml smtp_auth_file.yaml

接下来修改配置文件

修改 config.yaml 中

rules_folder: /etc/elastalert/rules

修改 my_rule.yaml中

smtp_auth_file: /etc/elastalert/rules/umu_smtp_auth_file.yaml

接下来就是创建systemd服务了

cd /etc/systemd/system

sudo touch elastalert.service

sudo vi elastalert.service

elastalert.service 内容

[Unit]

Description=elastalert

After=elasticsearch.service

[Service]

Type=simple

User=root

Group=root

Restart=on-failure

WorkingDirectory=/usr/local/dev/elastalert

ExecStart=/usr/bin/elastalert --config /etc/elastalert/config.yaml --rule /etc/elastalert/rules/my_rule.yaml

[Install]

WantedBy=multi-user.target

保存退出。

启动服务

sudo systemctl start elastalert

查看状态

sudo systemctl status elastalert

不出意外的话，你将看到绿色的Active: active (running)

至此，日志的异常监控服务就搭建好了，好的，收工！

---------------------

作者：梦得溪

来源：CSDN

原文：https://blog.csdn.net/pujiaolin/article/details/52252950

安装elastalert（按照官网的操作走）

先切换到自己的工作目录我这里是 /usr/local/dev/ git 克隆源码

git clone https://github.com/Yelp/elastalert.git

cd elastalert123安装

sudo python setup.py install1出错了 Running blist-1.3.6/setup.py -q bdist_egg –dist-dir /tmp/easy_install-Gc6gbe/blist-1.3.6/egg-dist-tmp-Ik7LL2 The required version of setuptools (>=1.1.6) is not available, and can’t be installed while this script is running. Please install a more recent version first, using ‘easy_install -U setuptools’.

(Currently using setuptools 0.9.8 (/usr/lib/python2.7/site-packages))

ok按照它提示的，，，执行：

sudo easy_install -U setuptools1再执行安装（sudo python setup.py install），又出错了

warning: no files found matching 'blist.rst'blist/_blist.c:38:20: 致命错误：Python.h：没有那个文件或目录 #include<Python.h>123这其实是缺少开发包，执行命令：

sudo yum install python-devel1再执行安装（sudo python setup.py install）到这里基本能安装成功。有时候会报一些包找不到的错误，可能是网络问题，重复执行安装命令即可（我在阿里云上遇到了）。

继续执行安装命令

pip install -r requirements.txt1找不到pip ，先安装pip并更新

sudo yum -y install python-pipsudo pip install --upgrade pip12安装完继续

sudo pip install -r requirements.txt1在elasticsearch中创建elastalert的日志索引

sudo elastalert-create-index1根据自己的情况，填入elasticsearch的相关信息，关于 elastalert_status部分直接回车默认的即可。如下所示：

Enter elasticsearch host: log.example.comEnter elasticsearch port: 9200Use SSL? t/f: tEnter optional basic-auth username (or leave blank): es_adminEnter optional basic-auth password (or leave blank):Enter optional Elasticsearch URL prefix (prepends a string to the URL of every request):New index name? (Default elastalert_status)Name of existing index to copy? (Default None)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py:734: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)New index elastalert_status createdDone!1234567891011121314151617181920212223243.创建配置文件

sudo cp config.yaml.example config.yaml

sudo vi config.yaml123根据自己的具体情况进行修改

比如我这里的elasticsearch是ssl 并需要用户登陆的，配置大致如下：

run_every: minutes: 1

buffer_time: minutes: 15

es_host: log.example.com

es_port: 9200

use_ssl: True

es_send_get_body_as: GET

es_username: es_admin

es_password: es_password

writeback_index: elastalert_status

alert_time_limit: days: 212345678910111213141516171819202122以同样的方式配置规则

cd example_rules/

sudo cp example_frequency.yaml my_rule.yaml

sudo vi my_rule.yaml12345这里就是根据具体的elasticsearch的信息进行配置

es_host: log.example.comes_port: 9200use_ssl: Truees_username: es_admines_password: es_password#name属性要求唯一，这里最好能标示自己的产品name: My-Product Exception Alert#类型，我选择任何匹配的条件都发送邮件警告type: any#需要监控的索引，支持通配index: logstash-*#下面两个随意配置num_events: 50timeframe: hours: 4#根据条件进行过滤查询（这里我只要出现异常的日志，并且排除业务异常（自定义异常））filter:- query: query_string: query: "message: *exception* AND message: (!*BusinessException*) AND message: (!*ServiceException*)"#email的警告方式alert:- "email"

#增加邮件内容，这里我附加一个日志访问路径alert_text: "Ref Log https://log.example.com:5601/app/kibana"#SMTP协议的邮件服务器相关配置（我这里是腾讯企业邮箱）smtp_host: smtp.exmail.qq.comsmtp_port: 25#用户认证文件，需要user和password两个属性smtp_auth_file: smtp_auth_file.yamlemail_reply_to: no-reply@example.comfrom_addr: no-reply@example.com

#需要接受邮件的邮箱地址列表email:- "user1@example.com"- "user1@example.com"1234567891011121314151617181920212223242526272829303132333435363738接下来创建smtp_auth_file.yaml

sudo touch smtp_auth_file.yamlsudo vi smtp_auth_file.yaml

##配置文件内容user: "no-reply@example.com"password: "password"123456现在可以简单测试一下配置文件是否正确

sudo elastalert-test-rule ./my_rule.yaml1如果有问题，就检查配置文件。然后启动运行一下试试：

sudo python -m elastalert.elastalert --verbose --rule my_rule.yaml1然后故意产生一下异常试试，比如

if(true){ throw new NullPointerException("测试日志异常监控服务");}123执行一下，一般没啥问题，邮件应该就能收到了，有问题就检查配置文件。

4.我是要监控服务，所以要让elastalert以服务的形式运行首先创建elastalert的配置目录和相关文件

sudo mkdir /etc/elastalertcd /etc/elastalert-- 复制配置文件

sudo cp /usr/local/dev/elastalert/config.yaml config.yamlsudo mkdir rulescd rules

-- 复制规则文件sudo cp /usr/local/dev/elastalert/example_rules/my_rule.yaml my_rule.yaml

-- 复制邮件用户认证文件sudo cp /usr/local/dev/elastalert/example_rules/smtp_auth_file.yaml smtp_auth_file.yaml12345678910111213接下来修改配置文件修改 config.yaml 中

rules_folder: /etc/elastalert/rules1修改 my_rule.yaml中

smtp_auth_file: /etc/elastalert/rules/umu_smtp_auth_file.yaml1接下来就是创建systemd服务了

cd /etc/systemd/systemsudo touch elastalert.service

sudo vi elastalert.service1234elastalert.service 内容

[Unit]Description=elastalertAfter=elasticsearch.service

[Service]Type=simpleUser=rootGroup=rootRestart=on-failureWorkingDirectory=/usr/local/dev/elastalertExecStart=/usr/bin/elastalert --config /etc/elastalert/config.yaml --rule /etc/elastalert/rules/my_rule.yaml

[Install]WantedBy=multi-user.target1234567891011121314保存退出。启动服务

sudo systemctl start elastalert1查看状态

sudo systemctl status elastalert1不出意外的话，你将看到绿色的Active: active (running)

转载于:https://www.cnblogs.com/bigben0123/p/10043132.html

你可能感兴趣的文章

集合体系

查看>>

vi命令提示:Terminal too wide